Evaluating System Parameters on a Dragonfly using Simulation and Visualization

The dragonfly topology is becoming a popular choice for building high-radix, low-diameter networks with high-bandwidth links. Even with a powerful network, preliminary experiments on Edison at NERSC have shown that for communication heavy applications, job interference and thus presumably job placement remains an important factor. In this paper, we explore the effects of job placement, job sizes, parallel workloads and network configurations on network throughput to better understand inter-job interference. We use a simulation tool called Damselfly to model the network behavior of Edison and study the impact of various system parameters on network throughput. Parallel workloads based on five representative communication patters are used and the simulation studies on up to 131,072 cores are aided by a new visualization of the dragonfly network.